Inference model construction method, inference model construction device, recording medium, configuration device, and configuration method

ABSTRACT

An inference model construction method obtains a distribution of control points pertaining to a reference state and a distribution of the control points pertaining to a defined state with respect to a defined representation model. The method extracts a first feature value based on the distribution of the control points pertaining to the reference state. The method machine-learns the distribution of the control points pertaining to the defined state while using, as a label, the first feature value, and constructs an inference model based on a result of the leaning performed with respect to a plurality of the defined representation models.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of International Patent ApplicationNo. PCT/JP2021/006206 filed on Feb. 18, 2021, the entire disclosures ofwhich are incorporated herein by reference.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an inference model construction method,an inference model construction device, a recording medium, aconfiguration device, and a configuration method, and relatesparticularly to a technique for making three-dimensional renderingrepresentations using two-dimensional images.

Description of the Related Art

Rendering representations using 3D models have recently becomemainstream in the technical field of computer graphics, includingelectronic games. This is because the time required for processing suchas repetitive rendering and lighting computation can be reduced whenframes are rendered at a plurality of consecutive time points whilemoving a single character or background, as in animation, for example.Especially, for interactive content such as electronic games, in whichcharacter’s motion or the like changes in real time in response tooperation input, rendering representations from various viewpointpositions and directions are enabled by preparing three-dimensionalmodels, animation data, or the like, in advance. In this type ofthree-dimensional graphics, commonly, a three-dimensional model isconstructed based on a plurality of two-dimensional images (cuts)prepared by a designer or the like, and rendering is performed byapplying textures to the model.

Meanwhile, three-dimensional graphics rendered by such textureapplication may give an impression different from that of an initial cutprepared by the designer or the like. Three-dimensional graphics arebasically for “correctly” rendering a three-dimensional model to whichthe texture is applied with respect to a specific viewpoint. It istherefore difficult to effectively reproduce a representation in aspecific line-of-sight direction, as cuts drawn in two-dimensionalimages do. For this reason, game contents or the like that prioritizethe attractiveness of representations unique to two-dimensional imagesand mainly use two-dimensional images on game screens also have acertain level of support.

Japanese Patent Laid-Open No. 2009-104570 discloses a renderingtechnique that enables representations of three-dimensional animation(three-dimensional representations) while maintaining the atmosphere andattractiveness of two-dimensional images drawn by a designer or thelike. Specifically, in Japanese Patent Laid-Open No. 2009-104570, atwo-dimensional image is broken down into parts, such as hair, eyebrows,eyes, and contour (face), and thereafter, a curved surface serving as areference is simply allocated to the contour part in accordance with theappearance of the two-dimensional image. Then, the two-dimensionalimages of the other parts are geometrically deformed and moved inaccordance with the spherical face of the contour that has been rotatedin correspondence with the direction of the face to be represented sothat various adjustments are also applicable. Thus, the designer’sdesired rendering representation from different directions is realizedwithout losing the impression of the original two-dimensional image. Inother words, unlike the method of simply applying textures forrendering, the method described in Japanese Patent Laid-Open No.2009-104570 employs a method of deforming a two-dimensional image so asto realize the designer’s desired rendering representation.

In the rendering technique described in Japanese Patent Laid-Open No.2009-104570, a three-dimensional representation in an angular rangeincluding a reference direction, such as a direction from the front, canbe generated by defining deformation of each part with respect to atwo-dimensional image drawn from the reference direction so as toachieve a rendering representation in a different direction (angle) fromthe reference direction. In other words, to generate a desiredthree-dimensional representation, the designer needs to definedeformation of each part relative to the angle (direction) at an edge inan angular range in which the designer desires to generate thethree-dimensional representation, and thereafter make fine adjustmentswhile checking whether the desired three-dimensional representation isgenerated at other angles (directions) in this angular range.

However, such definitions and adjustments may involve a great deal ofwork, and can raise concerns about the burden of these processes,especially on designers who are unfamiliar with that work.

The present invention has been made in view of the foregoing problems,and aims to provide an inference model construction method, an inferencemodel construction device, a program, a recording medium, aconfiguration device, and a configuration method that enable easygeneration of a representation desired by a designer using a method ofobtaining a three-dimensional representation by deformingtwo-dimensional images.

SUMMARY OF THE INVENTION

The present invention in its first aspect provides an inference modelconstruction method for constructing an inference model for inferringdeformation, in a defined state, of each part of a two-dimensional imageof a target object, with respect to a representation model for realizinga rendering representation corresponding to a state different from areference state of the target object by deforming the part of thetwo-dimensional image corresponding to the reference state, wherein thedefined state differs from the reference state, wherein therepresentation model is defined by defining deformation of each part ofthe two-dimensional image in at least one defined state, and isconfigured to realize a rendering representation corresponding to atleast a state between the reference state and the defined state, thedeformation of each part in the representation model is controlled by amode of a distribution of control points set for the part, the inferencemodel construction method comprises: obtaining the distribution of thecontrol points pertaining to the reference state and the distribution ofthe control points pertaining to the defined state with respect to adefined representation model, which is the representation model that hasbeen defined; extracting a first feature value based on the distributionof the control points pertaining to the reference state obtained in theobtaining; and machine-learning the distribution of the control pointspertaining to the defined state obtained in the obtaining while using,as a label, the first feature value extracted in the extracting, andconstructing an inference model based on a result of the leaningperformed with respect to a plurality of the defined representationmodels.

The present invention in its second aspect provides an inference modelconstruction device for constructing an inference model for inferringdeformation, in a defined state, of each part of a two-dimensional imageof a target object, with respect to a representation model for realizinga rendering representation corresponding to a state different from areference state of the target object by deforming the part of thetwo-dimensional image corresponding to the reference state, wherein thedefined state differs from the reference state, wherein therepresentation model is defined by defining deformation of each part ofthe two-dimensional image in at least one defined state, and isconfigured to realize a rendering representation corresponding to atleast a state between the reference state and the defined state, thedeformation of each part in the representation model is controlled by amode of a distribution of control points set for the part, the inferencemodel construction device comprises: at least one processor; and amemory configured to store instructions that, when executed by the atleast one processor, cause the at least one processor to function as: anobtaining unit configured to obtain the distribution of the controlpoints pertaining to the reference state and the distribution of thecontrol points pertaining to the defined state with respect to a definedrepresentation model, which is the representation model that has beendefined; an extraction unit configured to extract a first feature valuebased on the distribution of the control points pertaining to thereference state obtained by the obtaining unit; and a learning unitconfigured to machine-learn the distribution of the control pointspertaining to the defined state obtained by the obtaining unit whileusing, as a label, the first feature value extracted by the extractionunit, and constructing an inference model based on a result of theleaning performed with respect to a plurality of the definedrepresentation models.

The present invention in its third aspect provides a non-transitorycomputer-readable recording medium in which is stored a program forcausing a computer to execute the inference model construction method ofthe first aspect.

The present invention in its fourth aspect provides a non-transitorycomputer-readable recording medium in which is stored a program forcausing a computer to configure a representation model for realizing arendering representation corresponding to a state different from areference state of a target object by deforming each part of atwo-dimensional image corresponding to the reference state of the targetobject, by using the inference model constructed with use of theinference model construction method of the first aspect, wherein thedeformation of each part in the representation model is controlled by amode of a distribution of control points set for the part, the programcauses the computer to execute: input processing for obtaining thedistribution of the control points pertaining to the reference statewith respect to a configuration target object; first determinationprocessing for determining the first feature value based on informationobtained through the input processing; inference processing forinferring, with use of the inference model, the distribution of thecontrol points pertaining to the defined state of the configurationtarget object, based on the first feature value determined through thefirst determination processing; and output processing for configuringand outputting the representation model of the configuration targetobject, based on a result of the inference performed through theinference processing.

The present invention in its fifth aspect provides a non-transitorycomputer-readable recording medium in which is stored a program forcausing a computer to configure a representation model for realizing arendering representation corresponding to a state different from areference state of a target object by deforming each part of atwo-dimensional image corresponding to the reference state of the targetobject, by using the inference model constructed with use of theinference model construction method of the first aspect, wherein thedeformation of each part in the representation model is controlled by amode of a distribution of control points set for the part, the programcauses the computer to execute: input processing for obtaining thedistribution of the control points pertaining to the reference statewith respect to a configuration target object; first determinationprocessing for determining the first feature value based on informationobtained through the input processing; second determination processingfor determining the second feature value; inference processing forinferring, with use of the inference model, the distribution of thecontrol points pertaining to the defined state of the configurationtarget object, based on the first feature value determined through thefirst determination processing and the second feature value determinedthrough the second determination processing; and output processing forconfiguring and outputting the representation model of the configurationtarget object, based on a result of the inference performed through theinference processing.

The present invention in its sixth aspect provides a configurationdevice for configuring a representation model for realizing a renderingrepresentation corresponding to a state different from a reference stateof a target object by deforming each part of a two-dimensional imagecorresponding to the reference state of the target object, by using theinference model constructed with use of the inference model constructionmethod of the first aspect, wherein the deformation of each part in therepresentation model is controlled by a mode of a distribution ofcontrol points set for the part, the configuration device comprises: aninput unit configured to obtain the distribution of the control pointspertaining to the reference state with respect to a configuration targetobject; a determination unit configured to determine the first featurevalue based on information obtained by the input unit; an inference unitconfigured to infer, with use of the inference model, the distributionof the control points pertaining to the defined state of theconfiguration target object, based on the first feature value determinedby the determination unit; and an output unit configured to configureand output the representation model of the configuration target object,based on a result of the inference performed by the inference unit.

The present invention in its seventh aspect provides a configurationdevice for configuring a representation model for realizing a renderingrepresentation corresponding to a state different from a reference stateof a target object by deforming each part of a two-dimensional imagecorresponding to the reference state of the target object, by using theinference model constructed with use of the inference model constructionmethod of the first aspect, wherein the deformation of each part in therepresentation model is controlled by a mode of a distribution ofcontrol points set for the part, the configuration device comprises: aninput unit configured to obtain the distribution of the control pointspertaining to the reference state with respect to a configuration targetobject; a first determination unit configured to determine the firstfeature value based on information obtained by the input unit; a seconddetermination unit configured to determine the second feature value; aninference unit configured to infer, with use of the inference model, thedistribution of the control points pertaining to the defined state ofthe configuration target object, based on the first feature valuedetermined by the first determination unit and the second feature valuedetermined by the second determination unit; and an output unitconfigured to configure and output the representation model of theconfiguration target object, based on a result of the inferenceperformed by the inference unit.

The present invention in its eighth aspect provides a configurationmethod for configuring a representation model for realizing a renderingrepresentation corresponding to a state different from a reference stateof a target object by deforming each part of a two-dimensional imagecorresponding to the reference state of the target object, by using theinference model constructed with use of the inference model constructionmethod of the first aspect, wherein the deformation of each part in therepresentation model is controlled by a mode of a distribution ofcontrol points set for the part, the configuration method comprises:obtaining the distribution of the control points pertaining to thereference state with respect to a configuration target object;determining the first feature value based on information obtained in theobtaining; inferring, with use of the inference model, the distributionof the control points pertaining to the defined state of theconfiguration target object, based on the first feature value determinedin the determining; and configuring and outputting the representationmodel of the configuration target object, based on a result of theinference performed in the inferring.

The present invention in its ninth aspect provides a configurationmethod for configuring a representation model for realizing a renderingrepresentation corresponding to a state different from a reference stateof a target object by deforming each part of a two-dimensional imagecorresponding to the reference state of the target object, by using theinference model constructed with use of the inference model constructionmethod of the first aspect, wherein the deformation of each part in therepresentation model is controlled by a mode of a distribution ofcontrol points set for the part, the configuration method comprises:obtaining the distribution of the control points pertaining to thereference state with respect to a configuration target object;determining the first feature value based on information obtained in theobtaining; determining the second feature value; inferring, with use ofthe inference model, the distribution of the control points pertainingto the defined state of the configuration target object, based on thedetermined first feature value and the determined second feature value;and configuring and outputting the representation model of theconfiguration target object, based on a result of the inferenceperformed in the inferring.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a functional configuration of aconstruction device 100 according to an embodiment of the presentinvention.

FIG. 2 is a block diagram showing a functional configuration of aconfiguration device 200 according to the embodiment of the presentinvention.

FIG. 3A shows an example of a rendering representation (reference state)of a representation model according to the embodiment of the presentinvention.

FIG. 3B shows an example of a rendering representation (defined state)of a representation model according to the embodiment of the presentinvention.

FIG. 4A is a diagram for illustrating deformation of parts according tothe embodiment of the present invention (reference state).

FIG. 4B is a diagram for illustrating deformation of parts according tothe embodiment of the present invention (defined state).

FIG. 5A is a diagram for illustrating a control point group associatedwith deformation of parts according to the embodiment of the presentinvention (reference state).

FIG. 5B is another diagram for illustrating a control point groupassociated with deformation of parts according to the embodiment of thepresent invention (defined state).

FIG. 5C is a diagram for illustrating a control point group associatedwith deformation of parts according to the embodiment of the presentinvention.

FIG. 6A is a diagram for illustrating a difference in a movable rangebetween representation models according to the embodiment of the presentinvention.

FIG. 6B is another diagram for illustrating a difference in a movablerange between representation models according to the embodiment of thepresent invention.

FIG. 6C is yet another diagram for illustrating a difference in amovable range between representation models according to the embodimentof the present invention.

FIG. 7A is a diagram for illustrating normalization of a representationmodel according to the embodiment of the present invention.

FIG. 7B is another diagram for illustrating normalization of arepresentation model according to the embodiment of the presentinvention.

FIG. 8A is a diagram for illustrating second features value according tothe embodiment of the present invention.

FIG. 8B is a diagram for illustrating the second feature valuesaccording to the embodiment of the present invention.

FIG. 9 is a flowchart showing an example of construction processingexecuted by the construction device 100 according to the embodiment ofthe present invention.

FIG. 10 shows an example of a GUI for adjusting deformation of partsafter inference according to the embodiment of the present invention.

FIG. 11A is a diagram for illustrating processing for matchingdeformation of parts with the placement relationship set throughadjustment according to the embodiment of the present invention.

FIG. 11B is another diagram for illustrating processing for matchingdeformation of parts with the placement relationship set throughadjustment according to the embodiment of the present invention.

FIG. 11C is yet another diagram for illustrating processing for matchingpart deformation with the placement relationship set through adjustmentaccording to the embodiment of the present invention.

FIG. 12 is a flowchart showing an example of configuration processingexecuted by the configuration device 200 according to the embodiment ofthe present invention.

FIG. 13A shows an example of a data configuration of a representationmodel according to the embodiment of the present invention.

FIG. 13B shows an example of a representation model according to theembodiment of the present invention (texture information).

FIG. 13C shows an example of a data configuration of a representationmodel according to the embodiment of the present invention (stateinformation).

DESCRIPTION OF THE EMBODIMENTS Embodiments

Hereinafter, embodiments will be described in detail with reference tothe attached drawings. Note, the following embodiments are not intendedto limit the scope of the claimed invention, and limitation is not madeto an invention that requires a combination of all features described inthe embodiments. Two or more of the multiple features described in theembodiments may be combined as appropriate. Furthermore, the samereference numerals are given to the same or similar configurations, andredundant description thereof is omitted.

The following embodiment will describe an example of applying thepresent invention to a construction device that constructs an inferencemodel by machine-learning a plurality of defined representation models,and a configuration device that configures a representation model withuse of the inference model constructed by the construction device. Theinference model construction device and the configuration device in thepresent embodiment are described as separate devices. However, theimplementation of the present invention is not limited thereto. Thepresent invention may alternatively be implemented by a single deviceequipped with the functionalities of the construction device and theconfiguration device.

The “representation model” in the present specification will bedescribed as a data that can realize a rendering representationcorresponding to a desired state (i.e., generate a two-dimensional imagecorresponding to this state) by deforming each part of a two-dimensionalprovided as an image representing a reference state of a target object,and that can realize a three-dimensional representation by continuouslyshowing the process of the state transition. More specifically, torealize a rendering representation in a desired state, a representationmodel is data that contains information regarding deformation of atwo-dimensional image of each part defined for a state (defined state)that is different from the reference state, and that is configured to berealize a rendering representation corresponding to a transitionable(interpolatable) state at least between the reference state and thedefined state by using the technique described in Japanese PatentLaid-Open No. 2009-104570, for example. However, the representationmodel is not limited to one that realizes a rendering representation bymeans of interpolation between two defined states, and may alternativelybe configured to also realize a rendering representation that is notincluded in a range between these two states by means of extrapolation.

The “inference” in the present specification will be described as a termthat refers to deriving predetermined output based on a neural networkor the like that is constituted by an inference model, by givingpredetermined input to this inference model. Meanwhile, deriving a statethat represented by the input by applying a predetermined calculation tothe input without going through an inference model will be expressed as“estimation”.

Configuration of Construction Device

FIG. 1 is a block diagram showing a functional configuration of aconstruction device 100 according to the present embodiment. Here, theconstruction device 100 may be, for example, a server or the like thatis managed by a seller of a later-described editing application forconfiguring representation models, connects to a configuration device200 via a network (not shown), and is capable of providing a constructedinference model.

A control unit 101 is a control device such as a CPU, and controlsoperation of each block of the construction device 100. Specifically,the control unit 101 controls operation of each block by reading anoperation program for the block that is stored in a recording medium102, loading the read program in a memory 103, and executing it.

The recording medium 102 is a recording device such as a nonvolatilememory, which may be a rewritable ROM or the like, or an HDD that isremovably connected to the construction device 100. The recording medium102 records information such as parameters necessary for operation ofeach block, in addition to the operation program for the block of theconstruction device 100. Also, the recording medium 102 in the presentembodiment records a plurality of types of representation models used inmachine learning (i.e., representation models to be learned). The memory103 may be, for example, a volatile memory such as a RAM. Memory 103 isused not only as a loading area for loading programs or the like that isread from recording medium 102, but also as a storage area fortemporarily storing intermediate data or the like that is output duringoperation of each block.

A normalization unit 104 executes normalization processing fornormalizing the representation models to be learned so as to convergemachine learning performed by a learning unit 108. The details ofnormalization processing executed by the normalization unit 104 will bedescribed later. The normalized representation models may be stored inthe recording medium 102.

An obtaining unit 105 reads and obtains from the recording medium 102the representation models normalized by the normalization unit 104 inorder to machine learn the representation models. The normalizedrepresentation models obtained by the obtaining unit 105 are transmittedto an extraction unit 106, an estimation unit 107, and the learning unit108.

The extraction unit 106 extracts three types of feature values as firstfeature values according to the present invention, based on informationregarding reference states in the normalized representation models.Although the details of the three types of feature values will bedescribed later, the first feature values are information obtained byquantifying features of an external appearance of a target obj ect.

The estimation unit 107 estimates two types of feature values that serveas second feature values according to the present invention, based oninformation regarding the reference states and defined states in thenormalized representation models. Although the details of the two typesof feature values will also be described later, the second featurevalues are different from the first feature values in that the secondfeature values are information obtained by estimating and quantifyingfactors of deformation defined for the target object.

The learning unit 108 performs machine learning for one normalizedrepresentation model transmitted from the obtaining unit 105, based onthe feature values obtained regarding this representation model from theextraction unit 106 and the estimation unit 107. The learning unit 108constructs an inference model based on the learning results obtained bylearning a plurality of normalized representation models. Needless tosay, the inference model may be obtained in the form of a neuralnetwork, or may be obtained in any other form.

A communication unit 109 is a communication interface of theconstruction device 100 for communication with an external device. Thecommunication unit 109 connects to an external device and transmits andreceives data to and from this device via a network (wired or wireless),which may be a communication network such as the Internet, or may be alocal area network for connecting the devices.

Configuration of Configuration Device

Next, a functional configuration of the configuration device 200according to the present embodiment will be described with reference toFIG. 2 . Here, the configuration device 200 may be a terminal used by anend user in which an editing application for configuring representationmodels is executed, and is configured to obtain an inference model fromthe construction device 100 via a network (not shown). Note thatconstituent elements of the configuration device 200 of the presentembodiment that realize the same functionalities as those of constituentelements of the construction device 100 are distinguished by adding aprefix “configuration”.

A configuration control unit 201 is a control device such as a CPU andcontrols operation of each block of the configuration device 200.Specifically, the configuration control unit 201 reads an operationprogram for each block and a program of a later-described editingapplication for configuring representation models that are stored in aconfiguration recording medium 202, loads the read programs in aconfiguration memory 203 and executes them to control operation of theblock.

The configuration recording medium 202 is a recording device such as arewritable nonvolatile memory, such as a ROM, an HDD that is removableconnected to the configuration device 200, or the like. Theconfiguration recording medium 202 records information on parametersnecessary for operation of each block of the configuration device 200,in addition to the operation program for the block and the editingapplication program. The configuration recording medium 202 in thepresent embodiment also records the inference model constructed by theconstruction device 100. The configuration memory 203 may be, forexample, a volatile memory such as a RAM. The configuration memory 203is used not only as a loading area for loading programs or the like thatis read from configuration recording medium 202, but also as a storagearea for temporarily storing intermediate data or the like that isoutput during operation of each block.

A rendering unit 204 may be, for example, a rendering device such as aGPU, and generates an image (screen) to be displayed in a display regionof a display unit 220. In the configuration device 200 of the presentembodiment, the rendering unit 204 renders a two-dimensional image thatrealizes, with respect to at least a representation model that is beingedited, an external appearance of a target object indicated by thisrepresentation model, i.e., a rendering representation of this targetobject in a designated state, while the editing application is beingexecuted.

A display control unit 205 performs display control associated withdisplay of the screen generated by the rendering unit 204 on the displayunit 220. The display unit 220 may be, for example, a display devicesuch as an LCD, and may be integrated with the configuration device 200or may be an external device that is removable from the configurationdevice 200.

A setting unit 206 sets control points serving as a reference fordeformation of each part in a two-dimensional image of an object thatconstitutes a representation model (a configuration target object).Although the details will be described later, deformation of each partassociated with a change in the rendering representation realizes achange in the external appearance of the part by applying (mapping) thetwo-dimensional image of the part to a curved surface and changing thepositions of the control points set for this curved surface to changethe shape of the curved surface. For this reason, the setting unit 206sets a distribution of the control points in the reference state on theediting application, based on user input made according to a definitionof the reference state of the configuration target object. Although theuser is required to set the control points to define the reference statein order to realize a representation desired by the user in thedescription of the present embodiment, the implementation of the presentinvention is not limited thereto. The setting unit 206 may alternativelyset the control points based on image recognition, analysis of a layerconfiguration, or the like, without a user operation.

A determination unit 207 determines feature values associated with adefined state to be inferred by an inference unit 208 in order toconfigure a representation model of the configuration target object. Thefeature values determined by the determination unit 207 include thefirst feature values and the second feature values.

The inference unit 208 infers a distribution of the control pointspertaining to the defined state of the configuration target object,using the feature values determined by the determination unit 207. Theinference model constructed by the construction device 100 is used ininference performed by the inference unit 208.

A configuration unit 209 configures a representation model of theconfiguration target object, based on the result of inference by theinference unit 208. In other words, the configuration unit 209configures a representation model including the distribution of thecontrol points in the reference state that is set by the setting unit206 and the distribution of the control points in the defined stateinferred by the inference unit 208.

An operation input unit 210 is a user interface of the configurationdevice 200, such as a mouse, a keyboard, and a pen tablet. Upondetecting operation input made via any of various interfaces, theoperation input unit 210 outputs a control signal corresponding to theoperation input to the configuration control unit 201. The operationinput unit 210 notifies the configuration control unit 201 of theoccurrence of an event corresponding to the operation input.

A configuration communication unit 211 is a communication interface ofthe configuration device 200 for communication with an external device.The configuration communication unit 211 connects to an external deviceand transmits and receives data to and from this device via a network(wired or wireless), which may be a communication network such as theInternet or a local area network for connecting the devices.

Configuration of Representation Model

Next, a detailed configuration of a representation model used in thepresent embodiment will be described. In the description of the presentembodiment, an object to which a rendering representation is provided bya representation model is a character that at least includes the head,and a two-dimensional image representing an external appearance of thecharacter is displayed by presenting the representation model.

The external appearance of the character indicated by the representationmodel is constituted by two-dimensional images of various parts (face,left eye, right eye, nose, mouth, bangs etc.) of the character’s headconfigured for the forward orientation of the character, as shown inFIG. 3A. The two-dimensional image of each part is independentlydeformable, or is deformable in conjunction with deformation of otherparts. The deformation is controlled in accordance with the state of thedisplayed character. In other words, a representation model isconfigured to realize a rendering representation of a character in astate different from the reference state by deforming a two-dimensionalimage of the character facing forward, which is the reference state,rather than by switching to a two-dimensional image unique to each stateof the displayed character.

To realize a rendering representation in a state different from thereference state, the user needs to define a rendering representationcorresponding to at least one state different from the reference statein the representation model. The user can set a state (defined state) inwhich a desired rendering representation appears, in addition to thereference state, and define a rendering representation to appear in thethus-set defined state by deforming each part of the character in thereference state. Thus, the representation model can derive a deformationmode of each part and realize the corresponding rendering representationfor states in a range (hereinafter referred to as a “movable range”)specified by at least the reference state and the defined state. In thefollowing, a representation model in which deformation in at least onedefined state is thus defined and that can realize a renderingrepresentation of a state different from the reference state will bereferred to as a “defined representation model”. A representation inwhich deformation in no defined state has been defined and that canrealize only the rendering representation of the reference state will bereferred to as an “undefined representation model”.

For example, if a character facing forward, such as that in FIG. 3A, isdefined as a reference state, a rendering representation of thecharacter in an oblique orientation, such as that shown in FIG. 3B, canbe defined as a defined state by deforming the two-dimensional image ofeach part of the character in the reference state, and a renderingrepresentation corresponding to each state in the movable rangespecified between the reference state and that defined state can berealized. For ease of understanding of the invention, the states shownin FIGS. 3A and 3B are defined in the representation model described inthe present embodiment, and this representation model is configured torealize rendering representations of the head of a single characterturning in a yaw direction. Here, the turning in the yaw direction isnot limited to motion caused due to the head (or the entire character)rotating in the yaw direction without a movement of the character’sposition, such as the character shaking the head, but also includes amovement caused due to a change in appearance that occurs with thecharacter moving to the side (a mode in which the character is viewedfrom an oblique direction due to a shift from an optical axis(line-of-sight direction) of a camera that performs rendering). However,the implementation of the present invention is not limited thereto. Therepresentation model may alternatively include a plurality of types ofdefined states, such as an orientation change to another direction andtranslational movement, or a specific movement and a transition offacial expression, and may be configured to realize transition of therendering representation from the reference state to any of the definedstate, or from the reference state and a combined state of these definedstates.

In the present embodiment, deformation of two-dimensional images ofparts in the representation model is realized by applying thesetwo-dimensional images as textures to curved surfaces (including a flatsurface) and changing the shape of these curved surfaces. Morespecifically, to determine the shape of the curved surfaces, each curvedsurface is provided with a plurality of control points for specifyingits shape. A change in the shape of the curved surface and deformationof the applied two-dimensional image are realized by changing thedistribution of these control points. In the description of the presentembodiment, deformation of a two-dimensional image is realized byapplying the two-dimensional image as a texture to a curved surface andchanging the shape of this curved surface. However, the implementationof the present invention is not limited thereto. In other words,deformation of a two-dimensional image of a part may be simplycontrollable directly using the distribution of the control pointsprovided for the two-dimensional image, without using the indirectconcept of curved surfaces.

Curved surfaces and control points are set for two-dimensional images ofrespective parts in the reference state, as shown in FIG. 4A, forexample. Here, control points set for one part may be such that thenumber and distribution of the control points are determined inaccordance with the fineness required for a required deformationrepresentation of the part. For example, for the reference state and thedefined state of face part, a control point group is set at not onlyfour corners (vertices) of curved surfaces (rectangle) to which thesestates are applied, but also midpoints that divide sides of the curvedsurfaces into a predetermined number of sections (two horizontally andthree vertically in the shown example) and intersection points definedby connecting these midpoints, as shown in FIGS. 5A and 5B. The controlpoint group set at each of the vertices, midpoints, and intersectionpoints in FIGS. 5A and 5B may include a control point 501 thatcorresponds to an anchor point defining the position of a pointcorresponding to the curved surface, and control points 502, whichdefines a direction line of a curve (a bent line, a Bezier curve etc.)connecting adjacent control points (or any other control points on theirextensions), as shown in FIG. 5C.

A rendering representation of the character in a defined state isdefined by changing the distribution of these control points for eachpart, as shown in FIG. 4B, for example. Although the examples shown inFIGS. 4A and 4B illustrate the distributions of the control points(shape of curved surfaces) only for the character’s face, eyes, andmouth parts so that the distributions of the control points can beeasily observed, it is needless to say that deformation may also bedefined for any other parts as well.

After the deformation of the two-dimensional image of each part is thusdefined, a rendering representation of any state included in the movablerange defined by the reference state and the defined state can bederived by interpolation with use of the placement coordinates of thecontrol points in these states. More specifically, the state to berendered (target state) can be specified by the internal ratio betweenthe reference state and the defined state. Therefore, the placementcoordinates of the control points pertaining to the target state can bederived by weighting and adding the placement coordinates of the samecontrol points in the reference state and the defined state based on theinternal ratio. In other words, when rendering the character using adefined representation model, the target state can be specified by theinternal ratio in the movable range, and a rendering representation ofthe target state can be generated using the internal ratio as input.

Accordingly, for example, the representation model may include, inassociation with a model ID 1301 that uniquely identifies therepresentation model, texture information 1302 indicating variousinformation regarding two-dimensional images of various parts of thecharacter pertaining to the representation model, reference stateinformation 1303 indicating the distribution of control points definedfor various parts with respect to the reference state, and defined stateinformation 1304 indicating the distribution of control points definedfor various parts with respect to the defined state, as shown in FIG.13A. For ease of understanding the invention, one type of defined stateother than the reference state is defined in the representation model inthis embodiment, as mentioned above. Thus, only one type of definedstate information 1304 is included in the representation model data.However, the implementation of the present invention is not limitedthereto, and it is needless to say that the same number of types ofdefined state information 1304 as the number of defined states may beincluded. In this case, each piece of the defined state information 1304includes identification information that uniquely identifies a definedstate.

Here, for example, the texture information 1302 on the defined stateneed only include, for each of the parts constituting the character, arole ID 1312 indicating a role (right eye, nose, face etc.) of the partin the external appearance of the character, a two-dimensional image1313 of the part, detailed information 1314 storing various information,such as the size of the two-dimensional image of the part, andapplication-target curved surface information 1315 storing the size ofthe curved surface to which the two-dimensional image of the part isapplied, a list of set control points, and so on, in association with apart ID 1311 that identifies the part, as shown in FIG. 13B. Thereference state information 1303 and the defined state information 1304need only be configured to manage, for each part, a control point ID1322 that uniquely specifies the control point and placement coordinates1323 of the control point in the target state for each of the controlpoints provided to the curved surface pertaining to the part, inassociation with the part ID 1321 that identifies the part, as shown inFIG. 13C, for example.

Although a detailed description of the placement coordinates of controlpoints is omitted, it is needless to say that the placement coordinatesmay be absolutely specified relative to the center that is apredetermined origin provided for the entire character, or may berelatively specified around the center at predetermined coordinates of apart for which the control points are set or another part with which therelevant part is associated, for example.

Construction of Inference Model

Next, a description will be given of construction of an inference modelperformed by the construction device 100 of the present embodiment basedon machine learning of a plurality of representation models with defineddistributions of control points in the reference state and the definedstate. Although the details will be described later, a distribution ofcontrol points pertaining to a new defined state can be inferred for arepresentation model (undefined representation model) in which only thedistribution of the control points pertaining to the reference state isdefined, by using the inference model constructed by the constructiondevice 100.

Normalization of Representation Model

An inference model is constructed by the learning unit 108 performingmachine learning using a plurality of defined representation models astraining data. Although the movable ranges specified by therepresentation models do not necessarily coincide with each other, it isdifficult to machine-learn the distributions of the control pointsindicated by representation models in various modes as-is. Morespecifically, even if all reference states show characters facingforward as shown in FIGS. 6A to 6C, the mode of rendering representationin a defined state is not common to all the representation models.Therefore, simply learning the distributions of control pointspertaining to the defined states in these representation models does notlead to construction of a favorable inference model.

This is because the rendering representation of the defined statedepends on the representation method adopted by the designer and/or theuse application of the representation model.

Even if the representation models are configured to present turningmotion of the character’s head in the yaw direction as in the presentembodiment, the angle range in which the representation models canrealize the rendering representation of the turning motion depends onthe environment in which the representation models are used. Forexample, when a representation model is used in a chat system thatpresents a character in a bust shot whose behavior is controlledaccording to the user’s real-world motion, it is preferred to exaggeratethe character’s motion compared to the user’s real-world motion or toincrease the width of the character’s motion to make it stand out more.Therefore, the angular range required to achieve a renderingrepresentation in the representation model becomes wider. On the otherhand, when a representation model is used in adventure games thatpresent a substantially entire body (medium shot, full shot) of thecharacter, such as standing pictures, the range of motion of thecharacter is not very demanding. Therefore, the angular range requiredto realize a rendering representation in the representation model isnarrower.

In addition, there are only a limited number of cases in which,specifically, a rendering representation of a character viewed from anoblique angle is designed by strictly defining a specific numericalvalue of the turning angle from the front (viewing angle relative to theforward direction). In most cases, a rendering representation that isconsidered suitable by the designer’s sense is designed. Therefore, inan attempt to learn a certain representation model, it is difficult toderive the value of the turning angle indicated by the renderingrepresentation of the defined state in this representation model. On theother hand, even when a design is made by specifying the value of theturning angle, the representation method adopted by different designerscan vary, resulting in differences in the rendering representation(placement and deformation of parts) of the defined state even inrepresentation models for the same use.

In addition, in a representation model in which a change in appearancedue to the character moving to the side is specified so as to include arendering representation similar to turning in the yaw direction, partsare specified while being shifted in the direction of movement as wholein the rendering representation of the defined state. For this reason, adifference in rendering representation of the defined state can alsooccur depending on the presence or absence of translation, even betweenrepresentation models that realize a turning representation in the yawdirection.

Accordingly, since there are differences in feasible renderingrepresentation even between defined representation models, theconstruction device 100 of the present embodiment applies normalizationprocessing to defined representation models and then uses them asteacher data in order to obtain favorable learning results.

Here, the defined representation models may be normalized mainly throughtwo processes, namely “scale normalization” and “normalization of theamount of deformation”.

First, it is favorable that the size of a two-dimensional image of thecharacter used in the representation models is optimized in accordancewith the use of the representation models. That is, since the size ofthe range in which control points are distributed also differs betweenthe representation models, the normalization unit 104 normalizes thescale based on the placement of the character’s eye parts, which serveas first-type parts according to the present invention, in the referencestate.

First, the normalization unit 104 references the reference stateinformation 1303 in the representation model data, and derives aninterocular distance 701 in the representation model from the placementcoordinates of the left and right eye parts (e.g., the centercoordinates of the curved surfaces, the placement coordinates of thecontrol points allocated to the pupils), as shown in FIG. 7A. Thenormalization unit 104 then divides the values of the placementcoordinates 1323 in the reference state information 1303 and the definedstate information 1304 by the interocular distance, thereby configuringthe representation model data with the normalized scale.

Although scale normalization is based on the distance between the lefteye part and the right eye part in the description of the presentembodiment, the implementation of the present invention is not limitedthereto. It is needless to say that that scale normalization mayalternatively be based on the placement of any other parts.

Next, the rendering representation of the character presented by thedefined state differs between representation models, i.e., the amount ofdeformation in the defined state (from the reference state) differs evenfor the same part, as mentioned above. Therefore, the normalization unit104 normalizes the amount of deformation for the normalized models withthe normalized scale, based on the amount of movement of a control pointthat is set at the nose head of the character’s nose part, which servesas a second-type part according to the present invention. This isbecause, in a mode in which the representation models are configured topresent turning motion of the character’s head in the yaw direction, asin the present embodiment, the region of the character’s head thatshould be farthest away from the rotation axis of the turning is thenose head, and a movement (702) caused by the turning motion cancharacteristically appear as shown in FIG. 7B.

The normalization unit 104 configures the defined representation modeldata to be used as training data by adjusting the defined stateinformation 1304 in the representation model data with the normalizedscale so that the amount of movement of the placement coordinates 1323of the control point ID 1322 set at the nose head from the referencestate to the defined state takes a predetermined value (fixed value).Here, when M denotes the amount of movement of the control point at thenose head from the reference state to the defined state in therepresentation model data before the amount of deformation isnormalized, pf and p respectively denote the placement coordinates of acertain control point in the reference state and the defined state, andD denotes the amount of movement (fixed value) of the control point atthe nose head after the normalization, the placement coordinates p′ of acertain control point in the defined state in the representation modeldata after the amount of deformation is normalized can be derived by

p^(′) = (p − pf) × D/M

The amount of deformation due to the turning motion can thus benormalized for rendering representations of the defined states indifferent representation models. Representation models that absorb somedegree of difference in the movable range between representation modelscan be obtained as training data. In the following, a definedrepresentation model in which the normalization of the scale and theamount of change has been completed will be referred to simply as a“normalized representation model”.

The control point at the nose head may be identifiable by associatingthe role ID with information indicating the nose head when the controlpoint is defined, or may be identified as the control point with thelargest amount of movement of the horizontal coordinate among thecontrol points set for the nose parts, for example.

Although the amount of deformation is normalized based on the amount ofmovement of the control point set at the nose head in the description ofthe present embodiment, the implementation of the present invention isnot limited thereto, as with the scale normalization. The normalizationperformed to equalize the movable range between representational modelsmay be based on a specific control point included in a particular part.

Feature Values Pertaining to Normalized Representation Models

In machine learning of the normalized representation models(distributions of control points for the defined state), feature valuesthat appear in the normalized representation models are given as labels.In the normalized representation models in the present embodiment, thefirst feature values extracted from the reference state and the secondfeature values estimated based on the reference state and the definedstate are used as labels. These feature values will be described belowwith reference to the drawings.

The first feature values are information obtained by quantifyingfeatures (e.g., long face, large eyes etc.) of the character’s facepertaining to the normalized representation models. The extraction unit106 extracts, as the first feature values of the normalizedrepresentation models, the size of the two-dimensional image of eachpart, the size of the curved surface to which this two-dimensional imageis applied (which is, for example, obtained based on the placementcoordinates of control points set for four corners of the curvedsurface), and the center coordinates (e.g., the placement coordinates ofthe control point set for the center position), in the reference state.

Here, the content of the deformation to be represented can becharacteristically learned by including, in the feature values, the sizeof the curved surface to which the two-dimensional image is applied, inaddition to the size of the two-dimensional image of each part. If, forexample, the size of the two-dimensional image to be applied isrelatively small with respect to the size of the curved surface, theeffect of the deformation on the two-dimensional image is small even ifthe curved surface is deformed. Conversely, if, for example, the size ofthe two-dimensional image to be applied is relatively large with respectto the size of the curved surface, the deformation of the curved surfacehas a greater effect on the two-dimensional image. Accordingly, sincethe degree of deformation to be represented can vary depending on theratio of the size of the two-dimensional image of each part and the sizeof the curved surface to which the two-dimensional image is applied, theconstruction device 100 of the present embodiment can learn thecharacteristics of the deformation to be represented while more finelyclassifying these characteristics by including these sizes in the firstfeature value.

Meanwhile, the second feature values are not explicitly represented bythe normalized representation models, but are information quantified byestimating the type of motion that causes deformation of each part.

Here, assuming a use in which a distribution of control pointspertaining to the defined state in an undefined representation model isgenerated by using the results of inference with the inference modelconstructed by the construction device 100, it can be imagined that itis desirable to be able to change the obtained inference results inaccordance with the type of motion desired by the designer. In otherwords, when a distribution of control points pertaining to the definedstate in an undefined representation model is generated, it can beconsidered to be that improves convenience for the designer if theamount of deformation of parts caused by rotation of the character andthe amount of deformation of parts caused by a translation of thecharacter are inferred according to the factors (rotation andtranslation) of the required turning motion for the defined state, andthe distribution of control points can be determined based on the amountof deformation obtained by adding up these inference results. That is tosay, it is favorable that deformation of parts in the representationmodels to be used as training data are learned such that the deformationcan be separated into components generated by a rotation of thecharacter (rotational components) and components generated by atranslation of the character (translational components) and separatelyinferred.

As mentioned above, however, deformation of parts defined variesdepending on the use of a representation model, and the deformation modeof parts in the defined state also differs depending on therepresentation method adopted by the designer. It is, therefore,difficult to specify from the representation model alone informationregarding the motion that is assumed to have caused the deformation atthe design stage. In other words, it is difficult, for example, tospecify how much of deformation of parts pertaining to the defined stateis due to a rotation of the character and how much is due to atranslation of the character, or whether the deformation is not simplydue to motion but depends on the representation adopted by the designer.

Accordingly, for the normalized representation models to be used astraining data, the construction device 100 of the present embodimentdoes not separate the distribution of control points pertaining to thedefined state in accordance with factors (translation) and rotation) ofdeformation, but instead performs learning while using, as labels, thesecond feature values indicating the amount of translation and theamount of rotation that are estimated by the estimation unit 107 withrespect to the distribution of control points in a state where thesefactors are combined.

Information indicating the amount of translation is estimated based onthe control points set for a face part serving as a third-type partaccording to the present invention. The estimation unit 107 in thepresent embodiment derives, as information indicating the amount oftranslation, an amount of movement 801 of a control point set at thecenter of the face part from the reference state to the defined state inthe normalized representation models, as shown in FIG. 8A.

In the description of the present embodiment, the amount of translationin motion pertaining to the defined state is estimated based on theamount of movement of the control point set at the center of the facepart. However, the implementation of the present invention is notlimited thereto. For example, the estimation may alternatively beperformed based on the amount of movement of at least one control pointset for a face part, such as control points set for the four corners ofa curved surface pertaining to the face part, or may be performed basedon control points set for parts other than the face indicating theposition of the character’s head.

Meanwhile, information indicating the amount of rotation is estimatedbased on control points set for the left eyebrow, right eyebrow, andnose parts, which serve as fourth-type parts according to the presentinvention. The estimation unit 107 in the present embodiment derives, asan amount of movement 802 pertaining to the forehead, an average valueof the amount of movement of control points set at the inner ends of theleft and right eyebrow parts from the reference state to the definedstate, and derives, as information indicating the amount of rotation, adifference between the amount of movement 802 and an amount of movement803 of a control point set at the nose head from the reference state tothe defined state, as shown in FIG. 8B. This utilizes the length of apath drawn when a point placed in a three-dimensional space is rotatedabout a certain rotation axis varying depending on the distance from therotation axis (rotation radius). In other words, in the presentembodiment, it is assumed that a difference occurs in the amount ofmovement due to the rotation radius between the positions of theforehead and the nose head that are aligned with the median line of thecharacter and express unevenness of the character’s face, and thisdifference is obtained as information indicating the amount of rotation.

Similarly to the information indicating the amount of translation, theinformation indicating the amount of rotation is not limited to beingestimated based on the control points set for the left eyebrow, righteyebrow, and nose parts, but may alternatively be estimated based oncontrol points set for any parts that express unevenness in thecharacter’s head.

The learning unit 108 machine-learns the distribution of control pointsin the normalized representation models while using the thus-obtainedfirst and second feature values as labels, and constructs an inferencemodel based on the obtained learning results. Although the details willbe described later, the inference model constructed by the constructiondevice 100 of the present embodiment receives as input therepresentation models in which the reference state is defined (morespecifically, two-dimensional images corresponding to the referencestate and distributions of control points pertaining to the referencestate of a character configured by the representation models), andinfers and outputs a distribution of control points pertaining to thedefined state.

Construction Processing

A description will be given below, with reference to the flowchart inFIG. 9 , of specific construction processing executed by theconstruction device 100 of the present embodiment to construct aninference model based on the normalized representation models.Processing corresponding the flowchart is realized by, for example, thecontrol unit 101 reading a corresponding processing program recorded inthe recording medium 102, loading the program to the memory 103 andexecuting it. In the following description, the construction processingstarts upon an instruction to construct an inference model beingaccepted after a plurality of normalized representation models recordedin the recording medium 102 are designated, for example. Before theconstruction processing is executed, the normalization unit 104 executesnormalization processing in advance for the representation models to belearned, and the obtained normalized representation models are stored inthe recording medium 102.

In step S901, the obtaining unit 105 reads from the recording medium 102one normalized representation model (target model) from which featurevalues have not been obtained, out of the plurality of normalizedrepresentation models to be learned, under the control of the controlunit 101.

In step S902, the extraction unit 106 extracts the first feature valuesfrom information regarding the reference state in the target model,under the control of the control unit 101. More specifically, theextraction unit 106 extracts, as the first feature values, informationrepresenting the size of a two-dimensional image of each part, the sizeof the curved surface to which the two-dimensional image is to beapplied, and the center coordinates, based on the texture information1302 and the reference state information 1303 in the data of the targetmodel.

In step S903, the estimation unit 107 estimates the second featurevalues based on information regarding the reference state and thedefined state in the target model, under the control of the control unit101. More specifically, the estimation unit 107 estimates the amount oftranslation and the amount of rotation pertaining to deformation of theparts and obtains, as the second feature values, information indicatingthe estimated amount of translation and amount of rotation, based on thereference state information 1303 and the defined state information 1304in the data of the target model.

In step S904, the control unit 101 determines whether or not the firstand second feature values have been obtained from all of the pluralityof normalized representation models to be learned. The control unit 101advances the processing to step S905 if it is determined that the firstand second feature values have been obtained from all of the pluralityof normalized representation models to be learned. The control unit 101returns the processing to step S901 if it is determined that the firstand second feature values have not been obtained from all of theplurality of normalized representation models to be learned, i.e., thereare normalized representation models from which these feature valueshave not been obtained.

In step S905, the learning unit 108 machine-learns the distribution ofcontrol points pertaining to the defined state in the plurality ofnormalized representation models to be learned while using training datawith labels that are the first feature values extracted in step S902 andthe second feature values obtained in step S903, and constructs aninference model, under the control of the control unit 101. The machinelearning in this step is repeated until the difference (loss function)between the distribution of control points output by the inference modelfor the labels of the training data and the distribution of controlpoints in the training data converges.

In step S906, the learning unit 108 outputs an inference modelconstructed based on the results of learning performed for the pluralityof normalized representation models to be learned, and completes theconstruction processing, under the control of the control unit 101.

Thus, according to the construction device of the present embodiment, aninference model that enables inference of a deformation mode of eachpart can be constructed by using as training data a plurality ofrepresentation models in which various rendering representations in thedefined state are defined.

Configuration of Representation Model Using Inference Model

Next, a description will be given of configuration of a definedrepresentation model performed by the configuration device 200 of thepresent embodiment using the results of inference with the inferencemodel.

After obtaining the inference model constructed by the constructiondevice 100 of the present embodiment as described above, theconfiguration device 200 can infer deformation of each part for anundefined state. More specifically, if a two-dimensional image of eachpart and a distribution of control points pertaining to a curved surfaceto which the two-dimensional image is to be applied have been set forthe reference state of the configuration target object, a distributionof control points pertaining to a defined state can be obtained bygiving, as input, the first feature values extracted based on thereference state to the inference model.

Here, since the inference model is based on machine learning performedwhile using the first and second feature values as labels, appropriatesecond feature values need to be given to perform inference. However,the second feature values are information indicating the amount oftranslation and the amount of rotation regarding a distribution ofcontrol points pertaining to a defined state that is to be defined byinference, these parameters do not exist for an undefined state.Moreover, a deformation mode of parts pertaining to an undefined statedepends on a desire of a user (designer) who gives an instruction toexecute inference. However, there is no absolute measure for informationindicating the amount of translation and the amount of rotation either,and it is therefore unrealistic to make the designer designate specificnumerical values before inference. For this reason, the constructiondevice 100 of the present embodiment outputs, in advance, average valuesof the second feature values (amount of translation and amount ofrotation) pertaining to all the normalized representation models learnedto construct an inference model so that the configuration device 200 canperform inference using these average values as initial values.

In the description of the present embodiment, the average values of thesecond feature values in all the normalized representation models usedin learning are used as initial values of the second feature values wheninference is performed for an undefined state. However, theimplementation of the present invention is not limited thereto. Forexample, the second feature values may be derivable based onpredetermined features that appear in the reference state, or may be setin accordance with the use of the representation model.

Thereafter, inference is performed using the inference model based onthe first feature values obtained with respect to the reference state ofthe configuration target object and the initial values of the secondfeature values obtained from the construction device 100. Thus, adistribution of control points pertaining to a predetermined definedstate (which depends on the second feature values) of the configurationtarget object is output as the inference result. The configuration unit209 stores the distribution of control points of this inference resultin the defined state information 1304 in the data of the representationmodel pertaining to the configuration target object, thereby configuringthis representation model in a defined state.

Meanwhile, after setting the distribution of control points pertainingto the defined state, the two-dimensional image of each part can bedeformed to display the representation model, i.e., the defined state ofthe configuration target object can be visually presented. The designercan thus check whether the defined state is a desired renderingrepresentation. Accordingly, the editing application of the presentembodiment is provided with a graphical interface (GUI) with which,after the inference with the inference model, deformation of partspertaining to the defined state can be adjusted with respect to thedistribution of control points obtained by the inference, so that therendering representation of the defined state can be adjusted to changethe mode based on the initial values of the second feature values toanother mode.

As mentioned above, it is difficult to specify factors of deformation ofparts defined in a defined representation model. Therefore, the resultsof inference with the inference model constructed by the constructiondevice 100 of the present embodiment can include movement of controlpoints of translational components and rotational components. In thisediting application, in order to make it easy to adjust the inferenceresult to a rendering representation desired by the designer, theconfiguration device 200 separates the distribution of control points inthe inference result into translation components and rotationcomponents, and adjusts the degree of translation and rotation.Thereafter, the distributions of these control points are combined toobtain a distribution of control points pertaining to the adjusteddefined state. Here, the separation into the translation components andthe rotation components may be performed by, for example, deriving theamount of translation based on the amount of movement (from thereference state) of the center position of a specific part, such as aface part, and regarding, as the rotational components, the distributionobtained by subtracting the amount of translation from the placementcoordinates of all control points in the inference result, andregarding, as the translational components, a difference between theinference result and the rotation component, i.e., a distribution thatincreases the placement coordinates of all control points by the amountof translation.

The GUI may be configured so as to be capable of accepting adjustment ofat least either the amount of translation (translational level) or theamount of rotation (rotational level) with respect to deformation ofeach part in the defined state, as shown in FIG. 10 , for example. Uponadjustments (changes in slider values) being made via the GUI, thedefined state information 1304 in the data of the target representationmodel is changed, and the changed rendering representation of thedefined state is displayed on the display unit 220. If at least eitherthe amount of translation or the amount of rotation is adjusted, theplacement coordinates of the control points of a relevant part arechanged to values corresponding to the adjusted values with respect tothe distribution of control points of the translational components andthe distribution of control points of the rotational componentsseparated from the inference result, and the adjusted distribution ofcontrol points pertaining to the defined state is derived by adding upthe changed distributions of control points. Then, the defined stateinformation 1304 in the data of the representation model is updatedbased on the adjusted distribution of control points pertaining to thedefined state.

Here, when the amount of translation and the amount of rotation areadjusted for each part after the inference by using a GUI such as thatshown as an example in FIG. 10 , it is possible that the consistencybetween the parts fails after the adjustment. If, for example, theamount of translation is adjusted with respect to the distribution ofcontrol points for the character’s face part, the distribution ofcontrol points for the ear part is not changed, and therefore theplacement relationship will become inconsistent between the ear partsand the face part as shown in FIG. 11A. For this reason, constraintconditions are defined for the placement relationship between someparts, such as the ear parts and the face part, and the content ofchanges in the defined state information 1304 is controlled so that theplacement relationship is ensured before and after the adjustment.

More specifically, first, a grid 1101 is defined for the shape in thereference state of the curved surface to which a two-dimensional imageof a relevant part (which is the face part in the following description)is applied, as shown in FIG. 11B, for example. Here, the grid 1101 maybe for defining local deformation of the two-dimensional image appliedin a specific mode. However, the grid 1101 in the present embodiment isa concept for specifying a partial region (which is one rectangularregion subdivided by the grid 1101) of the curved surface that isrepresented by the distribution of control points. Informationspecifying which partial region a connection position between the facepart and the ear parts (e.g., the positions on the curved surface of theface part that the control point at the center of the ear part overlaps)belongs to and which position within that partial region the connectionposition is located, in the distribution of control points pertaining tothe defined state obtained by inference, is stored. The informationspecifying the positions in the partial region at which the connectionposition is placed may be constituted by the internal ratios (a, b) intwo side directions corresponding to a connection position 1102 in abounding rectangle of the partial region (a parallelogram whose oppositesides indicate diagonal directions of the partial region after theadjustment) when the corresponding partial region of the curved surfaceof the pre-adjustment face part is expanded to a two-dimensional planesuch as that as shown in FIG. 11C.

If at least either the amount of translation or the amount of rotationis adjusted with respect to the distribution of control pointspertaining to the face part after inference, the position specified bythe internal ratios (a, b) in the two side directions in the boundingrectangle of the corresponding partial region of the curved surface ofthe face of the adjusted face part may be specified, and thedistribution of control points for the ear part may also be changed sothat the ear part is connected at that position.

This ensures the placement relationship between some parts and reducesthe burden for adjustment on the designer.

Configuration Processing

A description will be given below, with reference to the flowchart inFIG. 12 , of specific configuration processing performed by theconfiguration device 200 of the present embodiment to configure arepresentation model using the inference model. Processing correspondingto the flowchart can be realized by the configuration control unit 201reading a program associated with the editing application recorded inthe configuration recording medium 202, loading the read program to theconfiguration memory 203 and executing it.

In the following description, the configuration processing starts whenediting work related to configuration of a representation model isstarted with respect to illustration data in which, for example,two-dimensional images of parts of the configuration target object areseparated, and a layer structure indicating the front-back relationshipduring rendering between the parts is defined. In the editingapplication corresponding to this configuration processing, in order topresent information necessary for the designer to configure arepresentation model of the configuration target object, the displaycontrol unit 205 executes processing to display a two-dimensional imagebased on illustration data or a rendering representation of a configuredrepresentation model in a designated state that is generated by therendering unit 204, as required in accordance with the frequency ofdisplay update on the display unit 220, for example. In the followingdescription, a description of display control processing that is notcharacteristic to the description of the present invention is omitted.

In step S1201, the setting unit 206 sets curved surfaces and controlpoints pertaining to the reference state for the two-dimensional imageof each part of the configuration target object, under the control ofthe configuration control unit 201. Specifically, the setting unit 206sets the curved surface to which the two-dimensional image is appliedand the control points for controlling deformation of this curvedsurface, with respect to the two-dimensional image of each part includedin the illustration data of the configuration target object, based onoperation input accepted via the operation input unit 210. After thesetting of the curved surfaces and the control points pertaining to thereference state has been completed, the setting unit 206 transmits thedefined information to the configuration unit 209, and configures dataof the representation model of the configuration target object thatincludes the transmitted information as the reference state information1303.

In step S1202, the determination unit 207 determines the first featurevalues and the second feature values with respect to the representationmodel of the configuration target object that is configured in stepS1201, under the control of the configuration control unit 201.Specifically, the determination unit 207 determines the first featurevalues based on the texture information 1302 and the reference stateinformation 1303 regarding the representation model, and determines thesecond feature values (initial values) from average informationregarding the training data associated with the inference model.

In step S1203, the inference unit 208 infers the distribution of controlpoints pertaining to the defined state in the representation model ofthe configuration target object, using the inference model while usingthe first and second feature values as input, under the control of theconfiguration control unit 201.

In step S1204, the configuration unit 209 stores information regardingthe distribution of control points pertaining to the defined state thatis inferred in step S1203 as the defined state information 1304 in therepresentation model data on the configuration target object, andcompletes configuration of the data in the representation model, underthe control of the configuration control unit 201.

In step S1205, the rendering unit 204 generates a renderingrepresentation of the defined state based on the configuredrepresentation model data, under the control of the configurationcontrol unit 201. The display control unit 205 then causes the displayunit 220 to display the generated rendering representation of thedefined state together with the GUI of the editing application relatedto the adjustment of each part after the inference, under the control ofthe configuration control unit 201.

In step S1206, the configuration control unit 201 determines whether ornot at least either the amount of translation or the amount of rotationhas been adjusted with respect to the distribution of control points forany part of the configuration target object. If it is determined that anadjustment has been made, the configuration control unit 201 advancesthe processing to step S1207, and if not, the configuration control unit201 advances the processing to step S1208.

In step S1207, the configuration unit 209 derives a distribution ofcontrol points of the changed part based on the adjusted amount oftranslation and amount of rotation, and updates the defined stateinformation 1304 in the representation model data, under the control ofthe configuration control unit 201. If a part for which the constraintconditions for the placement relationship has been set is adjusted, theconfiguration unit 209 accordingly changes the distribution of controlpoints of the corresponding associated part, and updates the definedstate information 1304.

In step S1208, the configuration control unit 201 determines whether ornot the editing work for the representation model of the configurationtarget object has ended. If it is determined that the editing work hasended, the configuration control unit 201 stores the data in therepresentation model in the configuration recording medium 202 andcompletes the configuration processing. If not, the configurationcontrol unit 201 returns the processing to step S1206.

Thus, the configuration processing of the present embodiment makes itpossible to configure a representation model of a configuration targetobject that includes a defined state indicating a desired renderingrepresentation with less workload.

Variation 1

In the above-described embodiment, inference with an inference model isexecuted only once when a distribution of control points for anundefined state is derived, in order to reduce the amount of computationin the configuration device 200. However, the implementation of thepresent invention is not limited thereto. For example, it is needless tosay that inference may be performed again for a relevant part using thesecond feature values based on information regarding the adjusted amountof translation and amount of rotation to obtain a distribution ofcontrol points pertaining to a changed defined state. In this case, itis considered that a more natural adjustment result is obtained than inthe mode of separating the distribution of control points obtained byperforming inference once into translational components andnon-translational components, separately adjusting these components andcombining them.

Variation 2

In the description of the above embodiment and variation, the secondfeature values are given when an inference model is constructed (machinelearning). However, the implementation of the present invention is notlimited thereto.

Giving the second feature values makes it possible to learn deformationappearing in a representation model to be learned after morespecifically classifying the deformation, and to obtain, as an inferenceresult, a distribution of control points pertaining to a defined statethat is more suited (more accurate) to an undefined representation modelfrom the constructed inference model. However, this is based on thepremise that a sufficient number of samples is available for therepresentation model to be learned. In other words, the more the numberof labels, the easier the representation model to be learned can belearned for each representation. However, it is possible that the numberof representation models to be learned for one combination of labelsbecomes smaller. It is, therefore, favorable that the absolute number ofsamples is large in order to avoid the problem of overlearning.

On the other hand, if the absolute number of samples is small, theprobability of overlearning can increase as the number of labelsincreases, and as a result, it is possible that the inference model doesnot yield favorable inference results. Therefore, when the presentinvention is implemented, an inference model may be constructed byperforming machine learning while giving only the first feature valuesas labels. In the configuration processing as well, only the firstfeature values may be given to obtain an inference result.

Variation 3

In the description of the above embodiment and variations, the firstfeature values include the size of two-dimensional images of parts.However, the implementation of the present invention is not limitedthereto.

The deformation mode of parts can vary depending on the ratio betweenthe size of the two-dimensional image of each part and the size of thecurved surface to which this two-dimensional image is applied, asmentioned above. However, the present invention can be realized withoutincluding the shape and the size of two-dimensional images as the firstfeature values, considering that when a representation model isconstructed, conventionally, a curved surface (distribution of controlpoints) in the reference state is defined in correspondence with theshape and size of the two-dimensional image of each part, and thatextremely large curved surfaces are not defined.

Variation 4

The above embodiment and variations has described an example method forconstructing an inference model by machine-learning representationmodels configured to realize rendering representations in which thecharacter’s head turns in the yaw direction. However, the implementationof the present invention is not limited thereto. In other words, therepresentation models to be machine-learned may include representationmodels configured to realize rendering representations in which thecharacter’s head turns in a direction other than the yaw direction. Forexample, to realize a three-dimensional representation of thecharacter’s head, a rendering representation of motion in which the headturns in the pitch direction as well as the yaw direction may also bedefined. In this case, inference models may be constructed respectivelyfor deformation of parts in the yaw direction and deformation in thepitch direction (one inference model for one dimension). Alternatively,one inference model may be constructed for deformation of parts in acombined state (an inference model that enables inference of adistribution of control points for two-dimensional deformation). Notethat, rendering representations that can be realized by representationmodels are not limited to such turning motion, as mentioned above.Therefore, the representation or the combination of representations forwhich an inference model is constructed may be changed as appropriate.

In the description of the above embodiment and variations, thedistribution of control points in an inference result is constituted bytranslational components and rotation components in order to configure arepresentation model that realizes a representation model in which thecharacter’s head turns in the yaw direction. However, the implementationof the present invention is not limited thereto. That is, factors thatdeform parts vary depending on the target rendering representation.Therefore, deformation other than translation is not limited torotation. In other words, in a mode in which a distribution of controlpoints in an inference result can be separated into components foradjustment, the adjustment target may be separable into translationalcomponents and non-translational components.

In addition, in the description of the above embodiments and variations,the construction of an inference model uses, as training data,representation models that include similar rendering representations(turning representation in the yaw direction). However, more favorableinference results can be obtained by additionally using, as trainingdata, representation models for the same use, representation modelsdesigned by the same designer, and/or representation models designed byadopting the same representation method, for example.

In the description of the above embodiments and variations, roles of theparts to be separated in representation models are defined. However, theimplementation of the present invention is not limited thereto. Althoughdefining the roles of the parts makes it possible to realize effectivemachine learning and favorable inference, the invention can also besimilarly implemented by inferring the roles from the placementrelationship between the parts, even if the roles of the parts are notdefined. In other words, the present invention does not require thatrepresentation models to be machine-learned have the same partconfiguration, or are in a state where the number of control points tobe allocated to each curved surface and the placement mode arepredetermined. Similarly, the present invention does not require thatthe reference state of a configuration target object for which a definedstate is inferred have a specific part configuration, or is in a statewhere the number of control points to be allocated to each curvedsurface and the placement mode are predetermined. In other words, theinference model need only be constructed by learning deformation of apart (or a part group) that is considered to be identical inrepresentation models that specify a similar drawing representation. Adistribution of control points that represent deformation may be learnedafter being transformed into a distribution with a specific fineness(resolution of representation). In contrast, when an inference resultbased on an inference model is used, a distribution of control pointspertaining to a defined state may be defined by adjusting the inferenceresult to the fineness of deformation that can be realized by thecontrol points set for the reference state of the configuration targetobject.

Other Embodiments

The invention is not limited to the foregoing embodiments, and variousvariations/changes are possible within the spirit of the invention. Theconstruction device and the configuration device according to thepresent invention can also be realized by a program that causes one ormore computers to function as these devices. This program can beprovided/distributed by being recorded in a computer-readable recordingmedium, or through a telecommunication line.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

What is claimed is:
 1. An inference model construction method forconstructing an inference model for inferring deformation, in a definedstate, of each part of a two-dimensional image of a target object, withrespect to a representation model for realizing a renderingrepresentation corresponding to a state different from a reference stateof the target object by deforming the part of the two-dimensional imagecorresponding to the reference state, wherein the defined state differsfrom the reference state, wherein the representation model is defined bydefining deformation of each part of the two-dimensional image in atleast one defined state, and is configured to realize a renderingrepresentation corresponding to at least a state between the referencestate and the defined state, the deformation of each part in therepresentation model is controlled by a mode of a distribution ofcontrol points set for the part, the inference model construction methodcomprises: obtaining the distribution of the control points pertainingto the reference state and the distribution of the control pointspertaining to the defined state with respect to a defined representationmodel, which is the representation model that has been defined;extracting a first feature value based on the distribution of thecontrol points pertaining to the reference state obtained in theobtaining; and machine-learning the distribution of the control pointspertaining to the defined state obtained in the obtaining while using,as a label, the first feature value extracted in the extracting, andconstructing an inference model based on a result of the leaningperformed with respect to a plurality of the defined representationmodels.
 2. The inference model construction method according to claim 1,wherein each part in the representation model includes a two-dimensionalimage of the part, a curved surface to which the two-dimensional imageis to be applied, and the control points specifying a shape of thecurved surface, and the first feature value includes informationindicating a center position and a size of the curved surface of thepart in the distribution of the control points pertaining to thereference state.
 3. The inference model construction method according toclaim 2, wherein the first feature value further includes informationindicating a size of the two-dimensional image to be applied to thecurved surface pertaining to each part in the reference state.
 4. Theinference model construction method according to claim 1, furthercomprising normalizing the defined representation model, wherein therepresentation model normalized in the normalizing is obtained in theobtaining.
 5. The inference model construction method according to claim4, wherein the target object includes a head of a character, and thenormalization includes: normalizing a scale of the distribution of thecontrol points pertaining to the reference state and the distribution ofthe control points pertaining to the defined state, based on a distancebetween two parts included in first-type parts constituting the head ofthe character; and normalizing an amount of deformation from thereference state to the defined state, based on an amount of movement ofcontrol points set for a second-type part constituting the head of thecharacter between the scale-normalized distribution of the controlpoints pertaining to the reference state and the scale-normalizeddistribution of the control points pertaining to the defined state. 6.The inference model construction method according to claim 1, furthercomprising estimating a second feature value based on the distributionof the control points pertaining to the reference state and thedistribution of the control points pertaining to the defined state thatare obtained in the obtaining, wherein in the learning, the secondfeature value estimated in the estimating is additionally used as alabel to perform the machine-learning.
 7. The inference modelconstruction method according to claim 6, wherein the second featurevalue includes information indicating an amount of deformation oftranslational components and an amount of deformation ofnon-translational components regarding deformation from the referencestate to the defined state.
 8. The inference model construction methodaccording to claim 7, wherein the target object includes a head of acharacter, the information indicating the amount of the deformation ofthe translational components in the second feature value is estimatedbased on an amount of movement of at least one control point set for athird-type part constituting the head of the character, between thedistribution of the control points pertaining to the reference state andthe distribution of the control points pertaining to the defined state,and the information indicating the amount of the deformation of thenon-translational components in the second feature value is estimatedbased on a difference between a plurality of control points set for afourth-type part constituting the head of the character, in an amount ofmovement between the distribution of the control points pertaining tothe reference state and the distribution of the control pointspertaining to the defined state.
 9. The inference model constructionmethod according to claim 8, wherein the fourth-type part is a part thatrepresents unevenness in the head of the character, and the amount ofthe deformation of the non-translational components is estimated basedon a difference in an amount of movement between a protruding portionand a recessed portion of the character.
 10. The inference modelconstruction method according to claim 8, wherein the third-type part isa part indicating a face of the character.
 11. An inference modelconstruction device for constructing an inference model for inferringdeformation, in a defined state, of each part of a two-dimensional imageof a target object, with respect to a representation model for realizinga rendering representation corresponding to a state different from areference state of the target object by deforming the part of thetwo-dimensional image corresponding to the reference state, wherein thedefined state differs from the reference state, wherein therepresentation model is defined by defining deformation of each part ofthe two-dimensional image in at least one defined state, and isconfigured to realize a rendering representation corresponding to atleast a state between the reference state and the defined state, thedeformation of each part in the representation model is controlled by amode of a distribution of control points set for the part, the inferencemodel construction device comprises: at least one processor; and amemory configured to store instructions that, when executed by the atleast one processor, cause the at least one processor to function as: anobtaining unit configured to obtain the distribution of the controlpoints pertaining to the reference state and the distribution of thecontrol points pertaining to the defined state with respect to a definedrepresentation model, which is the representation model that has beendefined; an extraction unit configured to extract a first feature valuebased on the distribution of the control points pertaining to thereference state obtained by the obtaining unit; and a learning unitconfigured to machine-learn the distribution of the control pointspertaining to the defined state obtained by the obtaining unit whileusing, as a label, the first feature value extracted by the extractionunit, and constructing an inference model based on a result of theleaning performed with respect to a plurality of the definedrepresentation models.
 12. A non-transitory computer-readable recordingmedium in which is stored a program for causing a computer to executethe inference model construction method according to claim
 1. 13. Anon-transitory computer-readable recording medium in which is stored aprogram for causing a computer to configure a representation model forrealizing a rendering representation corresponding to a state differentfrom a reference state of a target object by deforming each part of atwo-dimensional image corresponding to the reference state of the targetobject, by using the inference model constructed with use of theinference model construction method according to claim 1, wherein thedeformation of each part in the representation model is controlled by amode of a distribution of control points set for the part, the programcauses the computer to execute: input processing for obtaining thedistribution of the control points pertaining to the reference statewith respect to a configuration target object; first determinationprocessing for determining the first feature value based on informationobtained through the input processing; inference processing forinferring, with use of the inference model, the distribution of thecontrol points pertaining to the defined state of the configurationtarget object, based on the first feature value determined through thefirst determination processing; and output processing for configuringand outputting the representation model of the configuration targetobject, based on a result of the inference performed through theinference processing.
 14. A non-transitory computer-readable recordingmedium in which is stored a program for causing a computer to configurea representation model for realizing a rendering representationcorresponding to a state different from a reference state of a targetobject by deforming each part of a two-dimensional image correspondingto the reference state of the target object, by using the inferencemodel constructed with use of the inference model construction methodaccording to claim 6, wherein the deformation of each part in therepresentation model is controlled by a mode of a distribution ofcontrol points set for the part, the program causes the computer toexecute: input processing for obtaining the distribution of the controlpoints pertaining to the reference state with respect to a configurationtarget object; first determination processing for determining the firstfeature value based on information obtained through the inputprocessing; second determination processing for determining the secondfeature value; inference processing for inferring, with use of theinference model, the distribution of the control points pertaining tothe defined state of the configuration target object, based on the firstfeature value determined through the first determination processing andthe second feature value determined through the second determinationprocessing; and output processing for configuring and outputting therepresentation model of the configuration target object, based on aresult of the inference performed through the inference processing. 15.The recording medium according to claim 13, wherein the program furthercauses the computer to execute: display control processing for causing adisplay unit to display a rendering representation corresponding to thedefined state of the configuration target object, based on therepresentation model output through the output processing; acceptanceprocessing for accepting an adjustment of at least one of an amount ofdeformation of translational components or an amount of deformation ofnon-translational components, with respect to deformation in therendering representation corresponding to the defined state from thereference state; and change processing for, if the adjustment isaccepted through the acceptance processing, changing the distribution ofthe control points pertaining to the defined state of the outputrepresentation model, based on the adjusted amount of the deformation ofthe translational components and the non-translational components,wherein if the adjustment is accepted through the acceptance processing,the rendering representation corresponding to the defined state of theconfiguration target object is displayed in the display unit, based onthe representation model after the changing through the changeprocessing.
 16. The recording medium according to claim 15, wherein theprogram further causes the computer to execute separation processing forseparating the distribution of the control points pertaining to thedefined state inferred through the inference processing into adistribution of the translational components and a distribution of thenon-translational components, wherein in the change processing, thedistribution of the control points pertaining to the defined state ofthe output representation model is changed to a distribution of thecontrol points obtained by combining the distribution of thetranslational components and the distribution of the non-translationalcomponents that have been changed in accordance with the adjusted amountof the deformation of the translational components and thenon-translational components.
 17. The recording medium according toclaim 16, wherein a constraint condition regarding a placementrelationship is defined for at least some parts of the configurationtarget object, and in the change processing, placement positions of theat least some parts are changed so as to ensure the placementrelationship of the at least some parts before and after the adjustment.18. A configuration device for configuring a representation model forrealizing a rendering representation corresponding to a state differentfrom a reference state of a target object by deforming each part of atwo-dimensional image corresponding to the reference state of the targetobject, by using the inference model constructed with use of theinference model construction method according to claim 1, wherein thedeformation of each part in the representation model is controlled by amode of a distribution of control points set for the part, theconfiguration device comprises: an input unit configured to obtain thedistribution of the control points pertaining to the reference statewith respect to a configuration target object; a determination unitconfigured to determine the first feature value based on informationobtained by the input unit; an inference unit configured to infer, withuse of the inference model, the distribution of the control pointspertaining to the defined state of the configuration target object,based on the first feature value determined by the determination unit;and an output unit configured to configure and output the representationmodel of the configuration target object, based on a result of theinference performed by the inference unit.
 19. A configuration devicefor configuring a representation model for realizing a renderingrepresentation corresponding to a state different from a reference stateof a target object by deforming each part of a two-dimensional imagecorresponding to the reference state of the target object, by using theinference model constructed with use of the inference model constructionmethod according to claim 6, wherein the deformation of each part in therepresentation model is controlled by a mode of a distribution ofcontrol points set for the part, the configuration device comprises: aninput unit configured to obtain the distribution of the control pointspertaining to the reference state with respect to a configuration targetobject; a first determination unit configured to determine the firstfeature value based on information obtained by the input unit; a seconddetermination unit configured to determine the second feature value; aninference unit configured to infer, with use of the inference model, thedistribution of the control points pertaining to the defined state ofthe configuration target object, based on the first feature valuedetermined by the first determination unit and the second feature valuedetermined by the second determination unit; and an output unitconfigured to configure and output the representation model of theconfiguration target object, based on a result of the inferenceperformed by the inference unit.
 20. A configuration method forconfiguring a representation model for realizing a renderingrepresentation corresponding to a state different from a reference stateof a target object by deforming each part of a two-dimensional imagecorresponding to the reference state of the target object, by using theinference model constructed with use of the inference model constructionmethod according to claim 1, wherein the deformation of each part in therepresentation model is controlled by a mode of a distribution ofcontrol points set for the part, the configuration method comprises:obtaining the distribution of the control points pertaining to thereference state with respect to a configuration target object;determining the first feature value based on information obtained in theobtaining; inferring, with use of the inference model, the distributionof the control points pertaining to the defined state of theconfiguration target object, based on the first feature value determinedin the determining; and configuring and outputting the representationmodel of the configuration target object, based on a result of theinference performed in the inferring.
 21. A configuration method forconfiguring a representation model for realizing a renderingrepresentation corresponding to a state different from a reference stateof a target object by deforming each part of a two-dimensional imagecorresponding to the reference state of the target object, by using theinference model constructed with use of the inference model constructionmethod according to claim 6, wherein the deformation of each part in therepresentation model is controlled by a mode of a distribution ofcontrol points set for the part, the configuration method comprises:obtaining the distribution of the control points pertaining to thereference state with respect to a configuration target object;determining the first feature value based on information obtained in theobtaining; determining the second feature value; inferring, with use ofthe inference model, the distribution of the control points pertainingto the defined state of the configuration target object, based on thedetermined first feature value and the determined second feature value;and configuring and outputting the representation model of theconfiguration target object, based on a result of the inferenceperformed in the inferring.